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FOREWORD 



Converging factors point to a need to look for new pathways to vocational education excel- 
lence: the public's increased expectations regarding academic outcomes of education, heightened 
by a number of national reports; increased gradu&:ion requirements and declining vocational 
enrollments in many states; the emphasis in the Perkins Act on the need for strengthening aca- 
demic foundations; and business and industry requests that entry-level employees have a more 
thorough knowledge of the basic academics they will naed to apply in their vocational fields. 
Those concerned agree that students need to have stronger basic academic skills as they leave 
secondary education programs— stronger academic skills for graduation, for work, and for life. 

The National Center has sponsored diverse efforts dealing with basic skills in vocational edu- 
cation from research to development to dissemination. Much has been learned about vocational 
students' basic skills learning problems. In order to make connections between research and prac- 
tice, the National Center has, through synthesis and development, prepared an integrated pack- 
age for teacher use, reinforcing this information with practical applications gleaned from teachers' 
repertoires across the nation. The products in the package are aimed toward enabling vocational 
and academic teachers to strengthen the academic component of vocational programs through 
joint effort. 

The BASICS package provides resources in five focus areas: research findings, teaching tecn- 
niques, instructional materials, instructional strategies, and support roles. The resources are 
organized in three looseleaf guidebooks for flexible use, and an accompanying videotape provides 
an orientation to the topic and to the package. 

The Bridgets Guide orients administrators, counselors, teachers, employers, and families to 
the purpose and application of BASICS: individual roles are explained, resources identified, and 
implementation guidelines and strategies outlined in workshop format. Individual components to 
the guide are as follows: 

• Implementation Guide describes the Philosophy of BASICS and provides guidelines for 
implementing the program. 

• Support Roles for Basic Skills describes the role of administrators and counselors in a 
program for improving basic skills. 

• Primer of Exemplary Sttitegies provides teachers with examples of other teachers' suc- 
cessful efforts and diverse rpproaches. 

• Roadsigns from Research (posters and brochures) highlights key research findings of 
interest to teachers in strengthening basic skilis. 

Targeted Teaching Techniques provides vocational and academic teachers with assessment, 
planning, and management tools to improve students' basic skills. Individual components are as 
follows: 

• Technique for Management: Time for Learning lays foundations for more effective basic 
skills instruction through a study of the use of class time. 

v 




• Technique for Remediation: Peer Tutoring discusses the planning, implementation, and 
evaluation of peer tutoring programs to strengthen students' basic skills. 

• Technique for Computer Use: Software Evaluation describes a procedure for joint evalua- 
tion of educational software for basic skills instruction. 

• Technique for Individualization: The Academic Development Plan guides school staff 
through a systematic identification of individual student needs ancJ steps to meet those 
needs. 

• Techniques for Joint Effort: The Vocational-Academic Approach describes teaching 
techniques that vocational and academic teachers can use jointly to improve students' 
basic skills. 

Developing an Instructional Program provides teachers with practical and theoretical informa- 
tion on development or selection of appropriate applied basic skills instructional materials. Indi- 
vidual components are as follows: 

• instructional Materials Development discusses the prerequisites of materials develop- 
ment, alternative curriculum types, and guidelines for materials development and review. 

• Supplemental Instructional Resources identifies sources of basic skills instructional mate- 
rials availab'e for use with vocational students. 

• Instructional Assistance in Specific Basic Skills prepares vocational teachers to help stu- 
dents gain reading, writing, oral communications, and math skills. 

The National Center wishes to acknowledge the leadership provided to this effort by 
Dr. Robert E. Taylor, recently retired Executive Director. Appreciation is extended to the following 
individuals who served as a panel of experts to assist staff : n planning strategy and recommending 
document content: Eugene Bottoms, Consultant to the Southern Association of Colleges and 
Schools; : lichele Brown, Vocational Supervisor, Idaho Falls School District, ID; Alton Crews, 
Superintendent, Gwinnett County Public Schools, GA; Roger Faulkner, Instructor-Coordinator, 
Great Oaks Joint Vocational School District, OH; and Darrell Parks, Director, Division of Voca- 
tional and Career Education, Ohio Department cf Education. Appreciation also is extended to 
Robert Hartwell, Pioneer Joint Vocational School, Shelby, OH and to Ruth Gordon of The National 
Center for their critical review of the document. 

Special recognition is due the following National Center staff who played major individual 
roles in the development of the BASICS package: Richard J. Miguel, Associate Director for 
Applied Research and Development, and Michael R. Crowe, Project Director, for leadershp and 
direction of the project; Sandra G. Pritz, Senior Program Associate, Judith A. Sechler, Program 
Associate, and June Veach, Graduate Research Associate, for synthesizing and developing the 
documents; and Deborah Black for word processing the documents. Appreciation is extended to 
The National Center editorial and media services personnel for editorial review, graphics, and pro- 
duction of the documents. 



Chester K. Hansen 
Acting Executive Director 
The National Center for Research 
in Vocational Education 
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EXECUTIVE SUMMARY 



Research has shown that microcomputers are providing valuable assistance in improving the 
basic skills levels of students. As computer costs decrease, the capability of computers increases, 
and software availability expands, great potential exists for continued expansion of effective and 
economical solutions to a variety of student learning needs. 

Software programs in basic skills are available for many brands of microcomputers. Vocational 
courseware is available as well However, not all of thfese programs are of high quality. Evaluation 
of the software available is becoming organized, but in the final analysis, it is up to teachers to 
evaluate software for potential use by their students in computer-assisted instruction. 

This guide contains a compilation of suggestions and guidelines to assist secondary level 
teachers in planning to make the most effective use of computer-assisted instruction (CAI) 
through careful courseware evaluation. Vocational and academic teachers working together to 
develoo and implement guidelines for courseware evaluation are in a particularly good position to 
provide for students' vocational and academic needs. The potential is great because each teacher 
brings the perspective of a different teaching context and/or discipline. 

The guide is structured around a checklist that deals with three major tasks: plan for effective 
use of the computer,, establish guidelines for developing or evaluating software, and evaluate 
courseware. These tasks highlight the importance of evaluating courseware only on the basis of 
carefully constructed plans for CAI use and established evaluation guidelines. 

Three steps are discussed for teachers to plan for effective use o f the computer. These include 
the following: identify the benefits of CAI, identify appropriate tasks for CAI, and identify strategies 
for CAI. Information on these topics was compiled from two 1983 publications of the National 
Center, Microcomputers in Voc Ed: A Decision Guide (Zahniser, Long, and Nasman) and Micro- 
computer Software for Adult Vocational Education: Guidelines for Evaluation (Stone). 

Establishing guidelines involves the following steps: identify the learning objectives and tasks, 
determine teaching effectiveness, evaluate appropriate use of computer capabilities, identify man- 
agement possibilities, and evaluate documentation. Material for these topics was adapted from the 
Stone work previously cited. 

The task of actually evaluating courseware begins with identifying potentially suitable course- 
ware Then, to facilitate the evaluation, Courseware Evaluation: Form and Guide for Vocational 
and Technical Education (Chase. Gordon, and Makin, 1984) are presented. This material was devel- 
oped at the National Center to respond to the need for a mechanism for a detailed and compre- 
hensive courseware evaluation. 
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INTRODUCTION 



Interactivity is the current technological buzzword— and with good reason. The- 
orists have long agreed that the most effective learning occurs when the learner 
has a sense of control, is actively involved, gets immediate feedback, and has a 
high level of expectation about his or her ability to master the material . . . 
computer-assisted instruction (CAI) does just that. 

Chris Lee. "Adding the New Technology to Your Training Repertoire." 
Training: The Magazine of Human Resources Development. April 

.'982. 

The classification of an item of software as good or poor (useful . ' useless) is, in 
the final analysis, a professional judgment. It is our privilege to make such judg- 
ments and our fate to live with the results. All the world's questions, checklists, 
consultants, and research are means, not ends, in this decision-making process. 

B. Kansky, W. Heck, and J. Johnson. "Getting Hard-Nosed about Soft- 
ware: Guidelines for Evaluating Computerized Instructional Mate- 
rials." Mathematics Teacher. November 1981. 



Research has shown that microcomputers are providing valuable assistance in improving the 
basic skills levels of students. As computer costs decrease, the capability of computers increases, 
and software availability expands, great potential exists for continued expansion of effective and 
economical solutions to a variety of student learning needs. 

Software programs in basic skills are available for many brands of microcomputers. Vocational 
courseware is available as well. However, not all of these programs are of high quality. Evaluation 
of the software available is becoming organized, but in the final analysis, it is up to teachers (not 
necessarily computer teachers or computer directors) to evaluate software for potential use by 
their students in computer-assisted instruction. 

Definition of Computer-Assisted Instruction (CAI) 

Authors use a variety of terms wnen referring to the use of computers in instruction. Such 
terms include computer-based instruction (CBI); computer-assisted learning (CAL); computer- 
assisted instruction (CAI); and computer-managed instruction (CMI). Generally, the choice of 
terms is determined by whether the author is writing about an instructional delivery system or 
about a particular group of related instructional applications. 



The material in this and the next captor is adapted from Zahnisor. Long, and Nasman (1983) and Stone (1983) 
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Research 
Findings: 



In both theory and practice, it is often difficult to distinguish between CA! and CMI. 1 lis is 
because some of the existing courseware and software programs developed for ir<„ iructi' nal pur- 
poses are used both to deliver concepts and information to a student (CAI) and to diagnose the 
student's learning patterns and problems and prescribe remediation (CMI). In this guide, the term 
CAI will be useo as primary. 

The literature related to computer-assisted instruction contains a variety of definitions for the 
term. Some authors suggest that any instructional use of the computer is CAI, while others are 
much more specific in regard to the types of instructional applications that can be considered CAI. 
Inherent to all definitions, however, are the concepts that the computer can be he!pf ul in delivering 
instructional materials to students, and that the term CAI encompasses several different learning 
strategies. 

Frenzel (1980) describes and defines CAI as 

the process by which written and visual information is presented in a logical sequence to 
a student by a computer. The computer serves as an audiovisual device. The students 
learn by reading the text material presented or by observing the graphic information dis- 
played. The primary advantage of the computer over other audiovisual devices is the 
automatic interaction and feedback that the computer can provide. Multiple paths 
through the course material can betaken, depending upon the individual student's 
progress, (p. 86) 



A Joint Vocational-Academic Approach 

Vocational and academic teachers working together to develop and implement guidelines for 
software evaluation are in a particularly good position to provide for students' vocational and aca- 
demic needs. The potential is so great because each teacher brings the perspective of a different 
teaching context. 

A special concern for academic teachers' choice of software to strengthen basic skills arises 
partly because there has been relatively widespread discussion about CAI being especially suit- 
able for b»sic skills instruction. The agreed-upon potential in this area has resulted in a relatively 
large amount of basic skills software being produced. The challenge, then, is to select by careful 
evaluation from mis large supply. 

A special concern of the opposite nature exists for vocational teachers, since the market for 
vocational education software is smaller than that for general education software. This means that 
commercial software publishers will not be able to realize so large a profit from vocational soft* 
ware as they will from software prepared for the general education market. Because of this con- 
straint, there may be a smaller supply of "canned" or commercial software programs available for 
many vocational education courses. One source of information about available vocational educa- 
tion software is the database for Vocational Education Curriculum Materials (VECM), which 
includes almost 1.000 courseware listings. This database is maintained by the National Center for 
Research in Vocational Education (phone toll free within the continental United States, 
800-848-4815; in Ohio, 614-486-3655). A VECM listing describes the courseware and its availability. 
A few of these listings include evaluations done on a voluntary basis. These evaluations are sum- 
marized in ERIC (The ERIC Clearinghouse on Adult, Career, and Vocational Education). 
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Yet another issue is that the vocational curriculum must to some extent reflect the require- 
ments of local employers. Because of this, mass-produced, commercially prepared software will 
not. nor can it be expected to, satisfy the local curriculum needs of many vocational programs. 
Given this situation, the ability of the vocational community to aaapt software or to generate its 
own software becomes extremely important. 

It is clear that both vocational and academic teachers have an important role to play in estab- 
lishing the guidelines for developing or evaluating computer software and then in applying those 
guidelines. A joint effort should result in the use of software that is both vocationally and academi- 
cally effective. 

One way to implement such a joint effort : n evaluation is for a vocational teacher to team up 
with an academic teacher to estabiibh the guidelines. Then each can review the software 
independently— the vocational teacher focusing on specific vocational skills and the academic 
teacher focusing on the basic skills areas. The courseware selections would be made by joint 
review of their ratings. 

Additionally, vocational and academic teachers can offer much and profit much from the 
cross-fertilization that occurs through their joint software evaluation. As they discuss ways to 
evaluate software for their programs more effectively, they are likely to move toward greater 
vocational-academic integration of their courses. Learning gains in both vocational and academic 
courses should result from greater integration and coordination. 



What Follows 

The next section contains a compilation of suggestions and guidelines for planning to make 
the most effective use of CAI through careful courseware evak- lion. The checklist on the next 
page provides an overview of the items to be considered. Readers who are already familiar with 
particular items on the checklist may choose to skip over the discussion of those topics. The last 
pcrtion of the guide presents an evaluation form that can be duplicated for use in courseware 
evaluation. 
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PLAN FOR EFFECT5VE USE OF THE COMPUTER 




CHECKLIST-TECHNIQUE FOR COMPUTER USE 


□ 


Plan for effective use of the computer 




□ Identify the benefits of CAI 




□ Identify appropriate tasks for CAI 




□ Identify rategies for CAI 


□ 


Establish guidelines for developing or evaluating computer software 




□ Identify the learning objectives and tasks 




□ Determine teaching effectiveness 




□ Evaluate appropriate use of computer capabilities 




□ Identify management possibilities 




□ Evaluate documentation 


□ 


Evaluate courseware 




□ Identify potentially suitable courseware 




□ Use the courseware evaluation guide 




□ Use the courseware evaluation form 


Identify the Benefits of 
Computer-Assisted instruction 



Because of it& potential for interaction and for individualizing and personalizing instruction, 
the computer can be a powerful tool for strengthening students' basic skills. Us* of the computer 
has the effect of providing time that a vocationai (or academic) teacher may not be able to provide 
to help students deal with basic skill needs. Some teachors feel thot the use of computers detracts 
from interpersonal interaction in the learning process. However, use of a computer may actually 
enhance personal interaction between students ano the teacher since it can free teachers from 
basic instructional tasks and allow them to manage learning and to concentrate on students* other 

COnContS. 

► 
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As a learning tool for basic skills instruction, the computer has unique qualities. It has no feel- 
ings; it doesn't criticize, sympathize, become angry, interpret, or react except in the ways in which 
it has been programmed. This is at once a limitation and an asset. While the computer cannot 
sense a studoiu :> iac!< of understanding nor adjust its explanations to nuances of a student's inter- 
pretation of material, it can pose the very same question ten, fifteen, even a thousand times with- 
out any trace of impatience at the lack of a correct response. The computer gives all students the 
same reaction whether they are young, old, black, white, female, male, quick, slow, rich, or poor. 
This admittedly mechanistic quality of the computer has the advantage of being totally impartial 
and nonjudgmental. It can eliminate the anxiety often present in conventional classrooms, particu- 
larly with learners who have had negative experiences with education. Students who lack basic 
skills do not have to try to mask their inadequacies, because the computer is private as well as 
patient and consistent. Students using a computer do not have to waste energies over being 
"liked" (feeling approval) or "disliked" (feeling humiliated) and can, therefore, concentrate more 
completely on solving the problem or digesting the material presented. 

On the other hand, some students need the reassurance that can be provided only by human 
contact. Teachers using microcomputers to provide instruction should make sure that they or 
other resource persons are available to assist students and to offer guidance and support. 

While a computer makes no emotional judgments about the student user, its capacity for stor- 
age and retrieval of information does allow lor individualization of student use so that individuals 
not only interact with material at their ow . .<*vel and speed but also receive instant individualized 
responses. Through the microcomputer's interactive capabilities, CAI instruction adapts to the 
student's needs by processing the response to a qu^* % and coming back with whatever instruc- 
tion is required. When evidence is given that a stud mastered the concept or instructional 
objective, the lesson proceeds Individualization i avt , important feature of computer instruc- 
tion, since students have a variety of learning needs and learn at varying speeds. (To help monitor, 
use BASICS* Technique For individualization: The Academic Development Plan.) 

Unlike most other teaching methods or technologies, computers offer students the opportu- 
nity to follow a number of different learning directions or "paths " For example, if a student has 
already mastered part of a computerized lesson, he or she may choose to skip that section of 
instruction and "branch out" or proceed into new material. In other cases, the student may need to 
spend extra time reviewing difficult sections of the lesson— a process that is well facilitated by the 
microcomputer's backtracking or "looping" capabilities 

Clearly, therefore, the computer offers to both vocational and academic teachers a technology 
that is different from oth *r audiovisual media such as films, filmstrips, and audio and video tapes 
These latter items in particular allow only one path through the instructional material from start to 
finish As such, they do not permit nearly so many options for creating totally individualized and 
interactive instruction that the microcomputer does 

The one-to-one interaction between students and the computer, often enhanced by color, 
sound, touch, graphics, and animation, can keep students involved and motivated to a remarkable 
degree and can greatly extend their attention span This motivation can be especially important for 
students who have a passive or a negative approach to learning. 

In addition to providing motivation, the computer is capable of animating a process, such as 
the construction of an automobile engine, designing schematics and graphs, and presenting rou- 
tine drill and practice in the guise of a game. The computer can make many routine vocational 
learning tasks palatable as well as make less routine, otherwise inaccessible learning activities 
accessible to students. 




Any exposure to computers cau be a great benefit to students, since computer experience is 
becoming a requirement for much entry-level employment. In the huge white-collar field of office 
work, all but the most menial jobs now require the ability to use computers at some level of compe- 
tence, or soon will. Even entry-level file clerk jobs are affected as filing systems become databases. 
Computers have become so common that some people have suggested that computer literacy be 
conside, ed a basic skill. 

By working with computers in their day-to-day training, students develop competence in han~ 
dling menu-driven programs e..d learn the basic vocabulary of the technology. ,*hey come to 
accept computet /ork as routine. The apprehension many have about this new technology is 
reduced or disappears. Students develop a positive attitude toward the computer and feel better 
able to participate in a computerized world. 

Additional benefits that deserve at least brief mention include the computer's ability to— 

• provide embedded remedial instruction of which the student may not necessarily be 
aware. 

• provide enrichment material within the program; 

• track progress throughout the program, and, indeed, throughout a series of programs on 
varied material; 

• provide video and audio support via peripheral devices linked directly to the computer; 
and 

• provide a massive information base— either by direct display of the material itself or by 
directing the student to the appropriate medium. (Herriott, 1982) 

Obviously, the use of computers with approprate, well-designed software can dramatically 
and dynamically enhance instruction. 



Identify Appropriate Tasks for CAI 

Teachers' plans for use of the computer in instruction should stem from their instructional 
goals and objectives. Teachers need to analyze those goal* on a tcsk-by-task basis to determine 
which teaching techniques can best assist in reaching them. The computer offers a fresh and cre- 
ative approach to learning that makes it a candidate for consideration as a teaching tool for many 
tasks. 

Tasks that require drill can be made more interesting by computer, because problems can be 
randomized. Tasks that require routine computations can be performed more quickly— thus allow* 
ing students to examine higher-level concepts. Students can study or work with phenomena or. 3 
computer that in real life would be beyond them because of factors of time, cost, or physical 
danger. Tasks for which a large amount of material needs to be displayed lend themselves to com- 
puter instruction. 

Computers are most appropriate for tasks that can be explicitly defined. For example, they can 
be programmed to provide immediate feedback for specific answers and to advance students 
automatically to more difficult material when satisfactory competency is demonstrated. This indi- 
vidualization of feedback and pace is especially helpful in strengthening basic skills. 
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Computers cannot be used for every instructional task. Time at a computer is not an effective 
replacement for hands-on learning (e.g., a laboratory experiment). Neither should students be 
asked to read large amounts of material on a computer if the information can be provided equally 
well on paper. Often, psychomotor skills can be taught most effectively in ways other than by 
computer. 



Identify Strategies for CAi 

CAI can be implemented in the vocational or academic classroom in several different ways. A 
consensus has not yet been reached on <vhich of these modes are most appropriate for CAI. In any 
case, it is important to recognize some of the more common modes: 

• Drill and practice. This usa of the computer takes "advantage of the computer's tireless 
patience and ability to provide immediate feedback and reinforcement to prescribe, pro- 
vide, and monitor potentially very complex drill and practice activities which can be tai- 
lored to a student's individual needs." (Forman 1982, p. 44) 

Drill and practice are often discounted by the academic community, even though it is the 
method used to master specific academic areas. It is especially critical for students who 
come from homes where conventional curriculum topics are not supported. 

For such children, drill and practice take place only at school. These children rarely see 
or hear the basic rules of many subject areas used outside of school. They do not have 
the supervision, extra time, or support needed to practice the skills that stem from basic 
curriculum concepts. 

Students need to practice. They neef to practice what they have been taught so that they 
can use it with ease. Teachers simply do not have the time to give individualized drill and 
practice to each student in their classes, much less evaluate the results and provide feed- 
back. {McDaniels 1985, 146-7) 

• Tutorial. "Depending on the capabilities and the storage capacity of the computer system, 
[this use of the computer is a dialogue] between the learner and the designer of the 
educational program. The computer acts as a 'tutor' to teach the student concepts and 
skills. ..."(ibid.) 

Subjects like the basic skills can be taught well on computers because they have the 
advantage of being well defined, highly teachable through tutorial instruction. 
(McDaniels, 1985, 147) 

• Simulations. In this regard, the computer is used ". . . to simulate or generate environ- 
ments for the learner so [the student] can change variables and explore situations in a 
manner that might have been too expensive, too restricted by time limitations, too dan- 
gerous, or impossible to allow the student to explore in the real world." (ibid.) 

• Problem solving. "In this mode, the student uses the computer to solve problems that 
would take many hours or excessive computation to solve by hand. . . ." (Molettiere, 
Konsynski, and Stott 1980, p. 149) 
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• Inquiry. M ln this mode, a database of information is developed which the student can 
query for facts about a topic. An example of the inquiry mode is PIP, or "Product Informa- 
tion Package"— a database of product information thai marketing personnel can use to 
keep up-to-date on tariff status and product options." (ibid., p. 149) 

• Demonstration. The correct method of performing a task can be shown with the aid of a 
microcomputer, and the computer can repeat the demonstration over and over without 
boredom or frustration. 

• Writing. As a tool for creating text, the computer, coupled with word-processing software, 
can vastly simplify and speed up the tasks of editing. Rearranging sentences or whole 
paragraphs, altering a word or phrase, and correcting spelling and punctuation are ah fas- 
ter and far easier to cope with on the computer than with a typewriter or pencil. By reduc- 
ing this drudgery in writing, and particularly in rewriting, the computer makes the 
challenge of expressing oneseff "on paper" less intimidating. As a result, students may be 
able to learn writing skills more effectively and with less stress on a computer. (Stone, 
1983, p. 4) 

These modes of instruction plus other imaginative ways to implement CAI in the classroom 
offer ri^h possibilities for the strengthening of students' basic skills. Yet the computer cannot be 
viewbu as a panacea. And the use of the computer in the most effective way depends not only on 
the selection of appropriate tasks and strategies, but also on the use of quality software. The 
remainder of this guide suggests ways to ensure the use of quality software. 
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ESTABLISH GUIDELINES FOR DEVELOPING OR 
EVALUATING COMPUTER SOFTWARE 



Just as it has become clear that education can benefit tremendously from computer-assisted 
instruction, it is also true that care must be taken to avoid the pitfalls that go along with an enthu- 
siastic rush to embrace a new technology. One such pitfall would be the failure to question 
whether computer-assisted instruction is the best technique for a particular task. Another pitfall is 
the use of software from the large pool of mediocre (or worse) programs that have been produced 
as the industry has scrambled to respond to the market. 

As Lundeen (1983) notes, 

Educational software is available from a variety of commercial and noncommercial 
sources Library media specialists, school teachers, and students may also develop their 
own The area of microcomputer educational software is disorganized and largely cen- 
tered around a cottage industry lacking standards, quality control, and direction. Pro- 
grams vary widely in quality,, and the accompanying documentation (instruction for use) 
is often inadequate Costs vary from a few dollars to several hundred dollars for individ- 
ual software packages, and may not be related to the quality of the program (p 116) 

The following material specifies the issues that are important for the establishment of guide- 
lines for selection of quality software. 



Identify the Learning Objectives and Tasks 

The two most important questions to be answered in either the development of software or the 
evaluation of an existing program are, "What educational objective is addressed?" and "What is the 
student's learning task?" The more specific the objective, the easier it is either to evaluate or to 
plan a program. Once the objective and task have been identified, peripheral questions need to be 
asked, such as, "Is the computer appropriate for the accomplishment of this objective?" and "What 
category of computer software (e.o.,, drill, tutorial, simulation) can provide the best vehicle for 
reaching this objective?" 

At this point it might be advisable to look at the ways in which the particular task or objective 
has been approached in traditional teaching environments and ask, "In what way will computer- 
assisted instruction be more effective than established techniques for achieving this goal?" 

Developers can identify specific areas in which computer use could be constructive by asking, 
"What elements of the particular discipline or subject matter have traditionally posed teaching 
problems, and why?" If the problem is, for example, one of possible student injury or damage to 
expensive equipment caused by students' attempts to get experience in assembly or diagnosis, a 



The material in this chapter has been adapted from Stone (1993) 
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computer simulation is suggested. If the problem is that students are not motivated to drill them- 
selves in a required vocabulary or in basic skills, a lively computer game that demands accuracy 
and speed in the problem area may stimulate students to practice. 



Determine Teaching Effectiveness 

The entire program must be planned and evaluated to ensure that it provides an effective 
learning experience. This is the major and overriding concern. Even if the objective and task are 
well defined, and creative use is made of the computer's special capabilities, nothing will be gained 
unless the program meets other standards as well. 



Information Content 

The educational content should represent the best knowledge currently available in the area 
and should be correct. As any experienced software reviewer knows, errors creep into programs as 
they do into textbooks. A recent run-through of a word game, for example, revealed "concieve" 
given as the correct spelling of "conceive." An otherwise humorless drill in vocabulary introduced 
each lesson as "EXERCIZE." Careful review of the content is important. Presentation of wrong 
information only makes learning more difficult for students. 



Response Handling 

Another teaching consideration that is vital to the success or failure or a program is how 
incorrect student responses are handled. Some programs refuse to accept an answer in a different 
form and continue only to pose the question until the student has arrived at the programmed cor- 
rect answer. Such treatment could be a model of frustration. At the other extreme, the program will 
respond simply "INCORRECT" and proceed to the next question or section of the exercise, leaving 
the student without any knowledge of the answer desired. 

An example of ineffective response handling is the following: A mathematics drill posed the 
question "3/5 + 3/5." A student answer of "6/5" elicited a response of "INCORRECT." To a second 
try of "6/5," the computer responded, "Please reduce." Clearly, the computer had been pro- 
grammed to expect the answer "1 1/5," the correct answer in mixed number form. The student, 
however, had not been given the instruction to enter answers in mixed number form. More impor- 
tant, the answer "6/5" is not only correct but it cannot be reduced, although it may alternatively be 
expressed in mixed-number form. 

An effective option for handling an error is to allow the error and illustrate the consequences. 
In a paramedic training program that used simulation, the choice of "artificial respiration" rather 
than "tourniquet" for a slashed wrist could result in the death of the patient. The attractiveness of 
this approach is illustrated by the answer of one video arcade afficionado who was asked, "What is 
so great about video games?" He said, "You can die so many times." 



Goals 

While the actual goal of any instruction is student mastery of material, subsidiary goals can be 
introduced to make attainment of mastery more interesting. By way of example, the three pro- 
grams described below each purport to increase speed and accuracy of typing. 
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The first program is tutorial. The student is given information about where to locate the fingers 
of each hand on the keyboard and is then presented with sequenced drills. At the end of an exer- 
cise, a scors is awarded reflecting words per minute and errors made. The student is then advised 
to repeat the lesson or advance to the next drill. In this program, motivation to continue is sus- 
tained primarily by the student's desire for mastery, although the score may be motivational. 

The second typing program starts by asking the student to set goals both in words per minute 
(wpm) and permitted errors per line. A beginner may elect to try for 5 wpm with a generous allow- 
ance of errors. The more advanced typist may opt for 80 wpm with no errors. Progress through the 
sequence of drills is governed by whether or not the student-set goal is achieved. 

A third program provides z sequence of drills, but the format is that of an arcade game. Enemy 
craft approach the student's spaceship; to save the ship, the student must type letters, words, or 
phrases before the alien craft attacks. 

The first of these typing programs is likely to attract only students who are highly self- 
motivated. For those who are afraid of failure and easily discouraged, the second version may be 
more appropriate, hor others who might not have the self-discipline to expend time and energy on 
drill, the arcade game format might well be the most successful. 

Some people support the notion that learning is "serious business" and that devices to make 
learning fun are, therefore, suspect. Nevertheless, the attraction of games and the success of 
game-formatted learning experiences have led software producers to market an increasing number 
of educational games Research on the educational value of electronic games tends to support the 
instructional value of such formats (Malone 1981). Although very few educational programs 
created specifically for vocational training are yet on the market, in considering purchase of those 
that are available or the development of new software for vocational purposes, the game treatment 
should be examined for its relationship to the educational objective as well as for its motivational 
value. 

Is the gaming element merely a reward for successful performance? Does the correct answer, 
for example, simply provide the player with the opportunity to participate in the "game"— 
permitting the shooting of a basket in basketball or the spinning of a spinner to determine a move 
or is the format intrinsically related to the subject matter? Such a relationship exists in educational 
games in which the goal of speed and accuracy is directly releted to the pressure of "enemy" 
attack, which increases as the player becomes more adept. Another example is an estimation drill 
based on bowling in which the better the player's estimate, the more pins are knocked down. 

Both have benefits. In the former case, correct performance is encouraged because playing 
the game is an immediate goal. This approach has practical value and can be applied to a wide 
range of subject matter. 



Time Use and Timing 

How much student time must be spent on a program for the student to master the material? In 
judging or planning software, teachers should consider the time required. An automated account- 
ing program for bookkeeping students may consume sixty hours of student time. On the other 
extreme, as little as ten minutes a day spent on drill and practice can be effective. Simulations take 
varying amounts of time depending on the complexity of the problem, but most take more than a 
single forty-five minute period. When computer use will, in all probability, take more than one class 
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period, teachers should plan for the recording of students' progress throughout the program so 
that they need not start from the beginning on succeeding sittings. 

Timing within a program must also be considered. Is there a specific limit on the amount of 
time allowed for student response? Is that limit reasonable or, better still, can it be preset by the 
student or teacher? What happens if the time limit is exceeded? If there is no limit, does the pro- 
gram have any signal (such as a beep or flashing screen) to attract a user's attention to the fact 
that a response is required? 



Language and Instruction 

Software programs, like all educational materials, must be geared to the reading capability of 
the intended user. Programs to be used in basic skills instruction should have a clearly identified 
reading level. When programs are to be used with students who may have difficulty reading, sup- 
plementary vocabulary help can be provided through the computer's branching capabilities. 

Instructions for the operation of the program must oe geared also to the reading level of the 
intended user. If the user's reading level is a problem, one solution is to teach operation of the 
program orally and ask users to pass along the instructions. When appropriate to content, this 
works well and can provide vauable enhancement o* self-image to the user-teacher. Where written 
instructions are necessary, text should be displayed with sp.xe between lines and without crowd- 
ing the screen. Users must be allowed to read at their own pace. A mechanism, therefore, should 
be built in to allow them to advance to the next "page" by depressing a particular key. 

Finally, safeguards should be built into the program so that, if instructions are not followed, 
the program responds appropriately. Incorrect execution of instructions should not "break" the 
program (return it to the beginning) but should result in further instruction to the user on how to 
proceed. 



Testing Software 

It is essential to test a software package. Not until material has been field-tested over a reason- 
able length of time, can one be certain that problems have been eliminated. Results of field tests 
should be made available to those who wish to purchase the software. It is best to confer with 
someone at an institution that is already us j the software. 

Software should be purchased only from a marketing source that allows a reasonable exami- 
nation period. Fortunately, more and more software producers are offering to refund the purchase 
price if software is returned within a thirty-day period. 



Societal Issues 

In recent years much criticism has been leveled at tests and workbooks that are inherently sex- 
ist, racist, or propagandists. Instructional material reflects the prejudices of its authors. This is no 
less true of instructional software. The computer's interactive capability and its power to involve 
the user impose a responsibility on the purchaser or developer to be aware of any implicit values in 
a given piece of software. 
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One must at least ensure that computer response to user input is not derogatory or ego deflat- 
ing. The nonjudgmental quality of the computer is much proclaimed as one of its virtues, but soft- 
ware is only as nonjudgmental as its authors. 



Evaluate Appropriate Use of Computer Capabilities 

Some aspects of teaching effectiveness need to be carefully considered especially when the 
instructional vehicle is the computer, which has unique capabilities. These aspects are graphics, 
sound and speech, interaction, and randomization. All of these can be used effectively to support 
basic skills instruction. 



Graphics 

The extent to which the graphics treatment is integrated with the subject matter may be diffi- 
cult to judge, but it is not hard to perceive when the graphics overpower the learning objective, 
becoming the focus of the user's attention. Creating graphics is such fun that the programmer can 
become infatuated with the sight of the images, little realizing that the student will become bored 
and impatient with purposeless repetition or long sequences of pictures that require no response. 
Any graphics display that impedes the students' progress through the program or that consumes 
time with nonlearning and nonmotivational activities should be avoiaed. 

Graphics displays should not reward incorrect or inappropriate responses. An oft- cited exarr- 
p!e is a computer version of the familiar HANGMAN game in which students may deliberately try 
to misspell a word in order to view the dire, but visually intriguing, consequences. At the other end 
of the spectrum is the program that neglects the computer's ability to model or picture, relying on 
word-for-word duplication of some textual material. 

Some text is, of course, necessary in most programs. Care must be taken to ensure that such 
text, when presented on the computer screen, is clear and easy to read. Since regular computer 
characters are small and often fuzzy when reproduced on a screen, particularly if a television is 
used as a monitor, special alphabetics may be required. Care must also be taken to ensure suffi- 
cient contrast between characters and background. Green against blue may look lovely on a color 
monitor, but a user M ith a monochrome screen may see nothing at all 



Sound and Speech 

If sound is a feature of the program, similar criteria apply. Does ine sound serve a purpose, or 
is it merely a distraction? Is the program just as effective with the sound turned off? Can you, in 
fact, turn it off? In a classroom with a number of machines operating, too many bleeps, bangs, 
wooshes, and fanfares played at too high a volume could create seriously annoying distractions 
for students and teachers alike. 

Speech synthesis capability, not yet fully implemented on microcomputers, may nevertheless 
be necessary for special applications such as a substitute for text in a program aimed at the semi- 
literate. Some software producers have found ways to integrate tape recordings with computer 
software designed to drill spelling or a foreign language, or simply to introduce a neophyte to 
computer use. Such a circumvention of the speech synthesizer may be worth consideration if 
vocalization is a necessary part of the software. 
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Interaction 



By far, the most important and effective capability of the computer to motivate learning and to 
drill, remediate, teach, or model educational material is its capacity for immediate response to the 
user's input With computer-assisted instruction, each student can be steadily and actively 
engaged in the practice of a skill, the development of an idea, the exploration of a concept, or the 
evolution of a problem-solving strategy. Students need not sit waving their hands in the air for 
attention or wait days or even weeks between taking a test and receiving the results. It is unfortu- 
nate that more software does not take advantage of this capability. 

Programs that are structured like books equipped with automatic page-turning devices or film 
that rolls on regardless of the viewer's input should be avoided. In fact, it i? useful when reviewing 
or creating software to be aware of the percentage of time the user will be actively engaged and of 
the mechanisms built into the program to remind the student that some response is required. 
Building in student choices and decision-making can keep instruction lively. When developing 
software, cevelopers should maximize user interaction within the confines of the educational 
objective. 



Randomization 

Much educational software calls upon the computer's ability to randomize words, numbers, 
questions, and so forth. Suppose a test poses ten questions but draws on a list of a hundred or 
more items. The items for the particular drill or test are selected by the computer's random number 
generator, a built-in function of the computer. Randomness varies from model to model. If the 
software under consideration relies upon randomization, it might be well to run some diagnostic 
tests on the random number generator of the machine to be used. 



Identify Management Possibilities 

Teachers can reasonably expect a computer to aid in the management of instruction, that is, 
to keep individual student records, calculate individual and group averages, generate progress 
charts, and maintain and update other statistical information. Ideally, this kind of computerized 
record keeping would be integrated with a branching program that administers tests, assigns stu- 
dent work on the basis of the computer-evaluated test results, and conveys any and all results to 
the teacher on demand. This has, in fact, been achieved in the computer-managed instruction 
(CMI) packages marketed on mainframe and minicomputers, such as Control Data's PLATO pack- 
age. Such a complex integration of management and instructional functions, however, has not 
been satisfactorily achieved with microcomputers. 

Some management programs for microcomputers correlate student data, but teachers must 
regularly compile and feed in that data. Also, some sequenced curriculum programs test students 
and assign appropriate work, but currently these two functions have not been successfully corre- 
lated in microcomputer software. 

The record-keeping capability of most educational software is limited to tracking errors made 
during the use of the program and providing a summary on the screen at the end of the exercise or 
session. An option to print out this summary can be, and sometimes is, included. 
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In evaluating the management capability of microcomputer software, it should be kept in mind 
that, in the present state of the art, a barter situation exists: increased capacity for record keeping 
usually means decreased program flexibility. 



Evaluate Documentation 

"Documentation" refers to the body of written material accompanying a software or hardware 
package. Good software documentation includes the following: 

• Clear marking on the outside of the package of the make and model of microcomputer for 
which the software is intended. 

• Information on the copyright and licensing conditions of the individual package. 

• A list, again on the outside of the package, of all equipment needed for the proper opera- 
tion of the software including the memory requirements. 

• Step-by-step directions for activating the software. These should include instructions for 
the order in which each peripheral piece of hardware is to be turned on, and all other 
information necessary to ensure that an inexperienced user can make the program run. 

• Explanatory material relating to lesson content, including a description of a sample run- 
through. For the teacher this material should include an outline detailing typical class- 
room and/or student assigned use, instructional level, teacher options, and management 
and editing capabilities. Material for the student should include a menu of options, the 
goal of the program, and what, if anything, the student must contribute to the interaction. 

• If the material is to be edited by the teachers, instructions for editing designed for a per- 
son with little or no computer experience. 

• Supplementary worksheets and textual material if dictated by content. 

Summaries of previous evaluations of the software are valuable, as are reference sources. 
Some particularly complex software includes a "hotline" telephone number that teachers and/or 
students may call for information about operational problems they incur in using the software. 

Manuals accompanying software and hardware have been severely criticized for their 
abstruseness, for the use of unexplained technical language, and even for blatant errors. 
Obviously, documentation should be clear and accurate, but one should not assume that it will be. 

A final and important note: software (and hardware) should be fun to use. Through the use of 
computers, the learning process should not only be facilitated but also made more enjoyable. 
Before using a program in the classroom, educators should ask themselves, "Would I like to use 
this particular software to learn?" 
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EVALUATE COURSEWARE 



Because the supply of courseware is large and growing, the identification of potentially suit- 
able courseware to evaluate is an impottant time saver for teachers. On the next page is a reprint 
of an ERIC Digest with information about microcomputer courseware evaluation sources. 

The Microcomputer Courseware Evaluation Form and Guide that foliow the ERIC Digest were 
developed in response to a widely felt need in vocational and technical education for a means to 
determine the quality of the growing number of microcomputer instructional programs, or course- 
ware, available today. The form and guide provide the mechanism for a detailed and comprehen- 
sive courseware evaluation. These can be used by vocational and academic teachers who need to 
determine the quality of courseware and by developers who seek to produce high-quality 
courseware. 

As teachers seek to strengthen students' basic skills through CAI and as they review 
courseware f; - this purpose, they should be aware that some courseware provides basic skills 
instruction ovt -ly, perhaps labeled as communications (reading, writing, speaking, listening) 
mathematics, or science. In other courseware, basic skills instruction is embedded in vocational or 
other course materials. In the latter case, it might be advisable to use two copies of the form for the 
evaluation-one for the ratings relative to basic skills, and one for the overall course. 

Another point of differentiation is between courseware that is to be used for an entire group 
and that to be used for specific individuals. Courseware for a group must be suitable for the range 
ot learning needs and styles within :hat group. Courseware for an individual can be evaluated 
according to that person's learning needs and learning style. 
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MICROCOMPUTER COURSEWARE EVALUATION 

SOURCES 



The Dilemma: Too Much Choice 

More than 10.000 instructional software packages have been 
published for elementary and secondary schools— thousands in each 
major discipline. We know, from magazine and journal laments, 
that much of this courseware is of poor quality Faced with such 
an overwhelming supply, how can you increase the probability of 
purchasing good courseware for your school system? 

The Solution: Courseware Evaluation 

Fortunately, others have addressed this problem and have im- 
plemented a solution: evaluating large numbers of courseware 
packages. Thus, your school system can begin its task by identify- 
ing potentially suitable courseware through the evaluations available 
from one or more evaluation projects. After verifying that the con- 
tent of some of the courseware is appropriate to your needs, you 
can select these few programs for your own specialists to evaluate 
prior to purchase. 

Some Cautions 

Not all sources of courseware evaluations are equally reliable. 
Before depending on the conclusions reached by a particular evalua- 
tion service, learn what you can about the evaluation methods and 
criteria it employs. It is especially important that evaluations 
include: 

• critical appraisal of content accuracy. 

• in-depth consideration of the appropriateness and effectiveness 
of the instructional strategies employed, and 

• input from testing with students. 

Note, too, that some evaluation services also consider how closely 
a given courseware package matches a state or local curriculum 
This selection phase may not yield conclusions 'hat are valid for 
your school system. 

Useful Sources cf Courseware Evaluations 

Identified in this Digest are just a few of the most comprehen- 
sive and readily accessible sources of courseware evaluations 
Described below are reviews available from two major courseware 
evaluation projects and those published in two magazines Also 
highlighted are two sources that identify only high quality 
courseware (according to different criteria). Courseware evaluations 
are also available through clearinghouses established by several 
states and provinces, including Alberta. British Columbia. Califor- 
nia, Florida, Iowa. Maryland. Minnesota, New York, North 
Carolina. Texas, and Utah. 



MicroSIFT Courseware Evaluations. Holinagel. D C . & Weaver 
D. W. (Eds.). (1982-1985). Courseware evaluations, sets 1-16 Portland 
OR: Northwest Regional Educational Laboratory (ERIC Document 
Reproduction Service Nos. ED 226 765 (sets 1-51 KD 234 772 (sets 
6-8); ED 239 606 (set 9): ED 245 666 (set 10). ED 249 9 1 8 (sets 11-12) 
ED 260 710 (sets 13-U); see the April 1986 issue of Resources m 
Education (R1E) f or the ED number for sets 15 and 16 

Probably the most thorough and consistently reliable evaluations 
are published by the MicroSIFT project. Each courseware package 
is reviewed by at least three professionals and tested with students 
from the intended audience group. 

The synthesized evaluations include descriptions of the 
courseware (objectives, prerequisites, content and structure 
documentation, potential uses), appraisals of major strengths and 
weaknesses, and a check against more than 20 content instruc- 
tional, and technical criteria. The evaluation concludes with a sum- 
mary of the content; instructional and technical ratings: and an 
overall recommendation statement (highly recommend, recom- 
mend use with little or no change recommend use only if certain 
changes are made, or do not recommend! A modified format has 
been developed for science courseware that includes descriptions 
of science processes and concepts and three additional content and 
instructional criteria. 

These evalutions (issued periodically in sets of 75 to 90) may be 
obtained from the ERIC Document Reproduction Service icall 
1-800-227-3742. prices vary), or through the online Resources in 
Computer Education (RICE) database available through BRS (call 
MicroSIFT: 503-248-6800 Ext. 551) Some of the later sets contain 
evaluations of courseware that has been revised since an earlier 
MicroSU T evaluation was published. Sets 15 and 16 will be the 
last sets of evaluations to be published by the MicroSIFT project 
in the current form, but the project will continue to collect evalua 
tions and make them available through the RICE database 



EPIE Micro-Courseware FRO/FILES (1982-1985) Miaocomputei 
courseware pro/files & evaluations WMerMiP.NY EP1E Institute 
Available by subscription. 

A comprehensive evaluation service is provided by the Educa 
tional Product Information Exchange (EPIE) in cooperation with 
Consumers Urion (CU). Each Courseware PRO/FILE is syntnesiz 
ed from the review*, of two or more evaluators, who have usually 
tested the courseware with students 
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Every 3* or 4-page Micro-Courseware PRO/FILE and Evaluation 
contains a lengthy description and evaluation of content, teacher 
and student use, and management, often including photos of sam- 
ple screens. Instructional design a*id software design are also ap- 
praised under the general headings of goals *nd objectives, con- 
tents, methods and approach, and evaluation and management. A 
summary of the evaluation is provided on the first page, along with 
recommendations to the producer. These are accompanied by 
overall ratings of instructional design and software design (in earlier 
PRO/FILES) or a summary recommendation (in more recent 
PRO/FILES). 

Also published by EPIE Institute and Teachers College Press. Col- 
umbia University, is TESS: The Educational Software Selector (1985 
Edition. S59.95). This guide includes review citations and an in- 
dication of whether a review is favorable. 



Electronic Learning, Burroughs, R. (Ed.*. New York: Scholastic. 
Published monthly September, October, ax*J January through April; 
published bimonthly November/December and May/June. 

Each issue olthis magazine for teachers includes an entire sec* 
tion on the instructional applications of computers. In addition to 
discussing how computers can be used to help teach, for example, 
foreign languages or business education, contributors to that sec* 
tion provide comparative evaluations of several relevant courseware 
packages. The reviews, completed by a single evaluator. are not 
detailed, but do provide a concise overview of each program's 
strengths and weaknesses, along with seven ratings (instructional 
design, content, appropriateness, interest level, ease of use. sup- 
port materials, and overall value). Also included are the publisher's 
comments on the evaluations, if available. 



Educational Technology, LipsiU. L. (Ed.). Lnglewood Cliffs. Nj: 
Educational Technology. Published monthly. 



Every issue of this well-respected publication includes reviews 
of two or more courseware packages. Each evaluation is provided 
by a single professional, who judges the program against explicit 
criteria and who is required to test the courseware with one or more 



students representative of the intended audience The prose evalua- 
tion reports consist 01 a detailed description of the courseware 
package and a cntical evaluation of its strengths, weaknesses, and 
potential use. Publishers are given an opportunity to respond to 
the reviews. 



Mattas. L. L. (1985). Only the best The discriminating software gr.:;'~ 
for preschool-grade 12. Sacramento. CA: Education News Servicer 
ERIC Document Reproduction Service No. ED 256 294. 



For those who do not have the time to read hundreds of individual 
evaluations. Only the Best is an excellent resource. f ^ it. Mattas has 
identified 113 programs that received the most ag went on high 
quality among the 16 evaluation services whose courseware reviews 
she examines. \ one-page report is provided for each program 

Also included in Only the Best are brief descriptions of 189 ' rerrly 
qualifying" programs and 13 more that Mattas considers "worth 
looking at." (For courseware in these last two groups, you would 
certainly want to read one or two more thorough reviews before 
making a decision to select any of the packages.) 



N^<%d^dat^al'^o/nputtr Service (1985). Tne yellow book A 
parent's guidi^o e&cfitKnohy sfund'oourseware. Washington. DC. 
National Educatiqn Xssocfttioh.t * f 



Teachers/reviewers of the NEA Educational Computer Service 
examined 1.500 courseware packages from which they identified 
as being of high quality, the 272 programs described in The Yellow 
Book. The descriptions provided for each package are brief, giving 
an overview of the content and a summary of the evaluator' s com- 
ments and recommendations. Again, you would probably want to 
conduct your own evaluation of any of these packages you consider 

Also identified in this publication are subsets of the 272 programs 
that are available in versions for specific microcomputers, including 
many less popular models. 



This digest was prepared by Robin Taylor. Coor nator of Com- 
puter Education. University of Maryland Baltimore County 
Catonsville, MD 21228. December 1985. 



MThu pubi: * til wu pf f pt if d with funding from the Sttionil Institute 
ot Educ* * US Department of Eductoon under contract no 
N1E-4UO4S-0G0I The opinion* exprtMCd in this report do not necetMnh 
»efleci the poultoni or pohciet of NlE o> ED 
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COURSEWARE EVALUATION GUIDE 



Evaluation Form Explanation 

Part A 

Part A contains descriptive information about the courseware product and should be filled out 
as accurately and completely as possible The needed information may be located in the hard- 
copy documentation or within the program itself Part A can serve as an initial screening device to 
determine whether the courseware review should be continued, for example, if the courseware is 
not compa. ble with the hardware or instructional setting, it probably would not be worthwhile to 
continue the evaluation process. 

Part B 

Related evaluation criteria are organized into eight sections. Each section reoresents a cluster 
of criteria needed for courseware evaluation and selection. It is important to note, however, that 
another element must be added to these criteria in order to arrive at a suitable evaluation of the 
courseware* your own judgment. Although each criterion is an important indicator of quality, tne 
overall evaluation of the courseware depends on your analysis of these criteria in relation to your 
own needs, therefore, it is important to keep several points in mind when completing this part of 
tne evaluation form: 

• The criteria in the courseware evaluation form are numerous but not exhaustive and 
represent current knowledge and perceptions regarding courseware evaluation. As tech- 
nology advances, interpretations of what constitutes high-quality courseware may 
change. 

• An attempt has been made to present the criteria objectively. However, some criteria 
reflect a certain dec, ee of subjectivity and personal values (e.g., "Program promotes 
productivity"). 

• No relative importance is assigned to individual criteria. The value attached to individual 
criteria is situation-dependent, each user must weigh criteria in light of the situation. 

• The suggested evaluation procedure does not explicitly provide for observation of student 
use of the courseware. This could be included in an evaluation, however, at the judgment 
of the user. 

In completing Part B, the user should first decide which whole sections are applicable to the 

specific courseware being rc-viev ed and then mark each section either A for applicable or 

N/A *or not applicable Then a .esponse should be given for every criterion in each section 

marked A: 

• YES indicates that the criterion is fulfilled. 

• SOMEWHAT indicates that the criterion is only partially fulfilled. 



Pages 23 through 43 are reprinted .rom Chase, Gordon and Makin (1984). 
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• NO indicates that the criterion is not fulf!led within the program but should be 

• N/A indicates that the criterion is not fulfilled and does not need to be 

The COMMENTS column should be completed, at least for every item checked SOMEWHAT, 
to explain further why that rating was given When "Application" is checked for Program Mode 
under Instructional Setting in Part A of the courseware evaluation form. Section VIII should be 
completed along with any other applicable sections This is a separate section because of the 
importance of application programs in vocational and technical education and because they 
require a different set o f criteria for evaluation 

Parte 

The purpose of Part C is to provide a means of summarizing your ratings of the courseware 
being evaluated. Although a complete review using the entire evaluation form is recommended in 
certain circumstances Part C could be combined with Part A and used as a short evaluation or 
initial screening device 

Glossary 

Program is designed so that student progress is determined by the spe- 
cific answers given 

Combination of disk (or other medium of transfer) and the accompanying 
documentation and materials for instruction 

Thin, usually flexible, plate on which data or programs are stored 

The description and instructions for use of a program Documentation 
may be in hard copy or within the program itself 

Response of program to user input of information 

Either a single item or collection of mechanical or electronic items 
required for use of a microcomputer program Examples of hardware 
include monitors and printers 

The section of the computer where instructions and data are stored 

List of choices within a program from which the user makes selections 

Microcomputer unit of instruction that can stand alone 

The method or strategy used in the presentation of the subject matter 

A group of separate programs related to one another in that each pro- 
gram bears, in addition to its own title, a collective title applying to the 
group as a whole. 

Support Materials Items that support the activities of the persons using the program (e.g , 

student workbook). 
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Branching 

Courseware 

Disk 

Documentation 

Feedback 
Hardware 

Merrory 
Menu 
Program 
Program Mode 
Series 




Suggested Courseware Evaluation Procedure 



The following is a suggested procedure for evaluating vocational and technical education 
courseware It is intended for beginning courseware evaluators Experienced courseware evalua- 
tors and professional reviewers can follow the procedure as is or adapt it in accordance with their 
background and need The steps in the procedure are as follows. 



1 Review the evaluation guide and form 



2 Review the documentation found in the hard copy and in the program This will necessi- 
tate a cursory run-through of the program 

3 Complete Part A of the courseware evaluation form The user may not be able to com- 
plete all information requested in Part A Complete as many of the items as possible 

4 Determine the feasibility of continuing the courseware evaluation. If there is compatibility 
between the items completed in Part A and user needs, the eva'ualion process should 
proceed If there is incompatibility the evaluation process may be discontinued 

5 Run the program as a good student, making correct responses 

6. Rerun the program as a poor student, making incorrect responses. 

7. Complete Part B of the courseware evaluation form depending on the program mode 
checked If an application program is included in the courseware, complete Section VIII 
and any other applicable sections of Part B If the program mode is other than applica- 
tion, complete all applicable sections of Part 3, excluding Section VIII. 

8 Summarize your ratings in Part B by completing Part C of the courseware evaluation 
form. 



9. Decide if the courseware meets the needs of the students 



Figure 1 presents this suggested courseware evaluation procedure schematically 
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Begin Course- 
ware Review 

~t~ 

Review Evalu- 
ation Guide 
and Form 

v 

Review Docu* 
mentation 

(Hard Copy and 
in Program) 

T 

Complete Part A 
of the Evaluation 
Form 




Rerun Program 
As Poor 
Student 



tion Program 
Jncluded, 

YES 



NO 



Complete 

Section VIM 
of Part 8 



Use 


*Y£L< 


Courseware 





Complete 
Sections I Vil 
of Part B As 
Applicable 

tz: 



Complete 
Part C 




Figure 1 . Suggested courseware evaluation procedure 
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Courseware Evaluation Form Item Definitions 



Part A 



An explanation of each item in Part A of the evaluation form is provided on the following 
pages 



I IDENTIFICATION 
Program Title 

Series Title 
Vocational Area(s) 

Subject Area(s) 
Topic(s) 

Developing Agency 

Author(s) 
Programmer(s) 



Name of the specific microcomputer instructional program 
(e g , Introduction to Patterns) and date of development. 

Name of the microcomputer instructional program series of 
which the specific program is a part (e.g., Know Your Pattern is 
a series of programs). 

One or more vocational or technical areas for which the pro- 
gram could be used: agriculture, business and office, health, 
home economics, industrial arts, marketing and distributive 
education, and trade and industrial. 

A more specific content level under the vocational area (e.g.. 
textiles and clothing). 

Specific topic(s) covered in the subject area specified (e.g . 
pattern alterations) 

The organization, agency or individual producing the course- 
ware, address and phone. 

Writer(s) of the content of the program. 

Person(s) writing the content in programming language. 



II. HARDWARE REQUIREMENTS 



ERLC 



Microcomputer 

K Memory Required 
Medium of Transfer 
Programming Language 
DOS Specifications 
Other Specifications 
Peripherals Needed 



Type of microcomputer needed to run the program {e g . Apple 
Me). 

Amount of memory needed to run the program (e.g., 48K). 

Means used for storing the program (e g.. flexible disk). 

Language used to program the content (e.g , BASIC). 

Disk operating system required (e.g , 3.3). 

Any additional requirements in relation to hardware. 

Any add-on hardware units required to run the program (e.g., 
two joysticks). 
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Ill PROGRAM FEATURES 
Network Version Provided 

Multiple Copies Required 

Program Can Be Modified 

Program Protected 
Data Disk Needed 

Field-Test Data Available 



Program runs on a centrally located microcomputer and is 
relayed to numerous student terminals 

Disk must remain in disk drive during operation of the pro- 
gram; requiring multiple copies if the program is used by stu- 
dents simultaneously 

Teacher can exercise the "list" command (access the lines 
making up me program) to make additions, deletions, or 
alterations 

Program cannot be listed (e g , lines making up the program 
cannot be accessed) 

Data on file disk are required for the retrieval of information 
needed to run the program (e g , employees and salaries to 
generate a payroll). 

The results ot field testing are available to prospective us *rs of 
the program. 



IV INSTRUCTIONAL SETTING 
Program Mode 



Student Target Population 
Grade Level(s) 
Instructional Grouping 



Strategy or method used to present the content 

Application' Provides a service by performing a job (e g , 
spreadsheet). 

Drill and Practice* Provides repetition of information or skill 
previously acquired 

Educational Gaming: Presents facts in new interesting ways 
and provides for logical guessing 

Simulation* Presents real or imaginary events, compressing 
extended time to develop problem-solving skills in a safe 
environment. 

Tutorial Introduces new concept(s) and provides for mastery 
learning by giving immediate reinforcement 

Type(s) of student for which ihe program was developed. 

Educational level(s) for which the program is intended. 

Instructional grouping(s) of students with which the program 
can be used. If designed for group use. will the program stimu- 
late cooperative t, ompetitive interaction? 



Prerequisite Student Skills Competencies students muit have before using the program. 
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Documentation 



Student Support Materials 



Teacher Support Materials 



Correlated Materials 



Instructions for using the program Specify whether these are 
given in the program or in printed form 

Accompanying materials for student use (e g . haoduuis. 
workbooks). 

Accompanying materials for teacher use (e g , program guide, 
tests) 

The program and other instructional materials (e g . textbook) 
are complementary 



Estimated Time for Use Approximate time required to use the entire program 



V. AVAILABILITY 



Free, Loan, Duplica- 
tion. Sale, Rent 



Means by which the program may be obtained 



Copyright Restrictions 



Back-up Policy 



The program is copyrighted (i e . reproduction or distribution 
of the program is forbidden without approval). 

Vendor makes a second copy of the program available free or 
at a reduced cost 



Preview Policy 



Courseware may be previewed before purchase under certain 
conditions. 



Update Policy 
Contact 



Vendor provides revised versions of the program 

Name, address, and phone of organization, agency, or individ- 
ual from which the courseware may be obtained. 



Part B 

An explanation of each criterion in Part B of the evaluation form is provided on the following 
pages 

1. SUBJECT MATTER 

1 Subject matter is a timely practical component of the curriculum and reflects information or 
skills that can be used by students in their occupational fields. 

2. Learning outcomes are clearly identified for students Objectives are presented at the 
beginning or placed throughout the program to reflect the progressive order of the desired 
learning. 

3. Information is correct (e.g., graphs, text, statistics). 

4 Subject matter is organized to reflect the usual sequence of events (e g., simple to complex, 
chronological order). 
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5 Racial, ethnic or sex groups are neither overrepresented nor underrepresented There are 
no inaccurate or biased generalizations about the characteristics of these groups 

6 Vocabulary, readibihty level, difficulty of the material, and interest level are suited to the 
students 

7 The subject matter reflects the actual knowledge ard skills currently used in the occupa- 
tional area 

8 Subject matter is written and presented in a manner to engage and maintain students' inter- 
est in learning the concepts or skills 

9 Important ideas and concepts are reinforced (e g . by emphasis, repetition, questioning) 
These same ideas and concepts are synthesized in summary 

10 Microcomputer capabilities (e g . immediate feedback, untiring repetition) appear to provide 
one of the best ways of presenting the subject matter 



II. TECHNICAL PRESENTATION 

1 Program runs consistently throughout without glitches (e g , program does not stall) 

2 The program displays text, makes calculations, draws graphics, and gives feedback fast 
enough to maintain students' interest 

3 Information is displayed in a format that is well designed and uncluttered Text is not ob- 
scured by overlay of graphics 

4 Words are spelled correctly, grammar and ounctuation are accurate throughout the 
program. 

5 Instructions are consistent and unambiguous, complete, understandable directions are 
given for running the program 

6 When color is used, it does not detract from the intended purpose of the program (e g , color 
makes material more realistic and interesting) 

7 Audio is clear in tone and understandable The audio does not distract students from the 
educational impact of the program The program has an option to delete the audio when 
desired. 

8 Graphics, either still or animated, do not detract from the subject matter presented. They 
illustrate and add meaning to the material. 



III. STUDENT INTERACTION 

1 . Students can use the program without excessive assistance from the teacher. 

2 Program promotes active rather than passive involvement of students by encouraging think- 
ing and problem solving. 
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3 Students have conti j\ over the amount of time spent on each activity, thus individualizing 
the instruction to their specific needs 

4 A list of choices from which students can select is provided Easy access to this list is avail- 
able so :> udents can make other selections when desired 

5 The program offers a way of going back to make changes when a wrong answer or response 
is given 

6 Tne complexity of the type of response is based on the capabiMy level of the students (e g , 
excessive keyboardmg is not required if it has not been taught) 

7 The program is "crash-proof " It does not step or forfeit information when students either 
give wrong responses accidentally or try deliberately to make it fail 

8. When a student requests "Help." the program gives further instructions, reviews previous 
instructions, or provides assistances in progressing through the program 

9. The program provides students with the opportunity to exit when necessary (e.g.. class 
period ends before program is completed) and to reenter at point ended, rather than start at 
beginning of the program again. 

10 Students can change the order in which they go through the program This permits them to 
go back to review or pick up information not covered. 



IV. PROGRAM INTERACTION 

1 Program interacts as soon as student response is made (e.g.. informs student of accuracy 
of answers, presents further information, or explains previous information). 

2 If the wrong answer is given, the program provides further information or clues (e.g , 
numoer of letters in the correct word is provided.) 

3 Correct responses are recognized in a positive manner (e.g., student is complimented on 
correct answer). The program's response to incorrect answers is not so interesting that 
incorrect responses are encouraged. 

4 Students are not addressed in a derogatory manner (e.g., "You dummy") when incorrect 
answer is given. 

5. Program does more than merely review the material; it provides the reason that the answer 
is incorrect (e.g., "Answer B is wrong because . . ."). 

6 The student is not permitted to continue making incorrect answers indefinitely. It is not 
possible to arrive at the correct answer by the process of elimination. 

7. The type of positive reinforcement changes as the program progresses, since feedback 
such as "You're terrific" becomes tiresome when overused. 

8. Program offers activities based upon the student's responses. Branching offers alternative 
activities, with different levels of difficulty or interest. Looping is a repeat of the activity for 
review. 
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9 The type and content of the feedback are geared to student comprehension 



V STUDENT EVALUATION 

1 Evaluation included in the program (whether test items or performance type) is based on 
the stated student objectives and indicates progress toward attainment of the objectives 

2 The results of each student's performance on the evaluation are provided by the program 
(e g , test score, items correct, items wrong) This information is protected by a separate 
password for use by the individual student and the teacher, 

3 Program identifies for Individual students the items for which correct and incorrect 
responses were made This assists students and teachers in understanding what corrective 
measures need to be taken This information is protected by a separate password for use 
by the individual student and the teacher 

4 A composite view of class performance is given (e.g . average, range, percentiles) on the 
evaluation This information is protected by a separate password for use by the teacher 

5 Hard copy of both individual student and composite class results is available to the teacher 
to facilitate record keeping Hard copy of individual test results is available to the student 

6 Trie type of test item used (e »i , true-false, multiple choice, performance) is varied to 
reflect the best method of determining student attainment of objectives 

7 Test items are easy to understand Content and vocabulary are consistent with those in the 
subject matter presented. 

8. A data bank of test items provides the teacher with the capability of generating tests by a 
random sampling of items. 



VI. DOCUMENTATION 

1. The language, vocabulary, and organization of the material in the documentation are easily 
comprehended. 

2. All information is correct (e.g . graphs, text, statistics) 

3 Expected learning outcomes are listed. If particular skills are to be developed, they are 
specified. 

4 An explanation of the ideas and principles from which the program was developed is given 
5. The particular skills to be learned through using the program are stated 

6 The teacher is given specific suggestions on where and how to combine the program with 
the existing curriculum. 

7 Suggested follow-up activities geared to the students are given to reinforce the information 
presented. 
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8 Recommendations on where and how to use ail student materials are given 

9 All necessary information is provided so that teachers or students can run the program 
from start to finish regardless of prior experience. 



VII WORK BEHAVIORS 

1 Students are made aware of their competencies in relation to their intended occupations 
Individual strengths and weaknesses can be determined and used as guidelines for further 
development. 

2 Program pr r o<?nts all work as tasks to be approached and carried out in a conscientious 
manner R egardless of the nature of the work, students are always encouraged to "give it 
their best effort." 

3 Program encourages the achievement of maximum outcomes through the use of available 
resource? 

4 Positive behaviors are advocated for getting, performing, and keeping a job These behav- 
iors include dependability, punctuality, cooperation, and initiative. 

5 Program encourages students to solve problems and make decisions that have transfer- 
ability to their occupations and everyday lives. 

6 Emphasis is placed on "people skills*' — the ability to communicate and get along with 
people 

7 Students complete the program feeling that they have accomplished something. Equally 
important, the means of accomplishment leaves students feeling good about themselves 
and their ability to complete the task. 

8 Individual creativity is promoted through the opportunity to develop new ideas, products, 
or ways of performing tasks. 



VIII. APPLICATION PROGRAMS 

1. Program offers sufficient versatility and detail that the coverage and complexity of the pro- 
gram can be changed to meet the specific needs of the students using it. 

2. Specific commands or instructions to enter and manipulate data are logical in nature and 
simple to use. 

3. Process required to change data (frequently numbers) is simple to understand and easy to 
use. 

4. Information being usgci in the program can be corrected or changed at any time without 
having to rerun the entire program. 

5. All fields and va r inbies necessary to perform the task are available, or the program is adapt- 
able so the necessary variables and fields can be added. 
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Program provides the same answer or outcome each time, so that one can depend on its 
accuracy 



7 Program performs the task it is supposed to do 

8 Supplementary information or data source is provided to jse in learning to run the program 

9 Program provides for printer use when hard copy of the resulting information is 
advantageous 

10 The sequence in moving from one operation to another is easy to understand and 
implement 

1 1 Program is either bundled (designed to be compatible with other application programs) or 
integrated (developed specifically to be combined with other particular application 



12 Tutorial program presents the concepts and information needed in learning to operate the 
application program 



programs) 



PartC 



An explanation of each item in Part C of the evaluation form is provided as follows. 



SUMMARY COMMENTS 



Describe the advantages of this particular 
courseware If possible, compare it with other 
courseware reviewed. 



Describe the disadvantages of this particular 
courseware If possible, compare it with other 
courseware reviewed. 



Explain different ways the courseware might 
be used in learning situations, both in the 
classroom and in informal settings. 



2 SUMMARY OF SECTION 
EVALUATIONS 



Summarize the rating of the courseware by 
section of Part B of the evaluation form. 



3 FINAL RECOMMENDATION 



Give a final overall rating of the courseware for 
instructional use and a brief explanation of 
why that rating was given. 



34 



ERjC 







Features of the Evaluation Form 


The Microcomputer Courseware Evaluation Form on the following pages— 


• 


is comprehensive, covering all areas o # vocational and technical education, 


• 


may be adapted to meet specific instructional needs, 


• 


can be used with all types of courseware. 


• 


is flexible in that only applicable sections need to be used, 


• 


provides space for written comments in addition to checked ratings, 


• 


allows Part A, Description, and Part C, Summary, to be corrbmed and used as a cur- 
sory screening evaluation if desired, and 


• 


has an accompanying guide to assist in the use of the form. 
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COURSEWARE EVALUATION FORM 



NOTE" If you are using th»s form for the first 
time, read the instructions in the accompany- 
ing Microcomputer Courseware Tvaluation 
Guide 



Evaluator . 
Position _ 
Date 



Part A: Courseware Description 

In the following sections, record descriptive information about the courseware that you are 
evaluating 

I. IDENTIFICATION 

Program Title Date 

Series Title 



o 

o 

e 

2 
<* 
S 

01 

* 
o 

O 



Vocational Area(s) . 



Subject Area(s) 

Topic(s). 

Developing Ager ,y _ 



Street or P O. Box 
City. 



. State_ 



Zip. 



. Phone( 



) — 



Author(s). 



Programmers) 



II HARDWARE REQUIREMENTS 
Microcomputer* 



(brand/model) 

K Memory Required 

(number) 

Medium of Transfer (include number of each) 1 

— Tape cassette Flexible disk 

— ROM cartridge 8" Flexible disk 



. Other. 



(specify) 



Programming Language 
Other Specifications. 



DOS Specifications. 



Peripherals Needed (check all that apply): 

Color monitor 

One disk drive 

Two disk drives 

Plotter 

Game paddle(s) 

Joystick(s) 



Modem 

Mouse 

Printer 

Graphics tablet 

Light pen 

Voice/sound 

instrument 



Clock 

Video disk 

Toucn screen 

Ten-key number 

pad 

Other 



(specify) 



•NOTE 1 Provide the above ^formation for any additional hardware on which this program 
can be used 
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III PROGRAM FEATURES (check al! that apply) 

Network verston provided 

Multiple coptes required 

Program can be modified 

IV INSTRUCTIONAL SETTING 

Program mode (check all thai apply) 

Application 

Drill and practice 

Student Target Population (check all that apply) 



i Program prntorlnri 

Data disk needed 

Field-test data available 



Educational 
gaming 
. Simulation 



Tutorial 

Other __ 



(specify i 



Regular 

Disadvantaged 

Grade Level (check all that apply) 

K-6 9-10 

7-8 11-12 

Instructional Grouping (check all that apply) 

individual 

Small group (up to 4) 

Largi? group (4 or more) 

Prerequisite Student Skills (specify) 



Handicapped 

Limited English 



. 13-14 
.Adult 



Bilingual 

G«fted 



, Higher 
Education 



competitive interaction 

cooperative interaction 



Accompanying Materials (specify types) 
Documentation 



Student support materials _ 
Teacher support maienals. 
Correlated materials 



Estimated Time for Use 

V. AVAILABILITY 
Free 



(copies) 



.Loan. 



Sale S_ 

Rent S_ 



(time) 



(time) 

Duplication (requestor supplies disk) 

Copyright Restrictions (explain) . 

Back-up Policy (explain) 

Preview Policy (explain) 

Update Policy (explain) 

Contact 



Street or P O Box 
City 



.State- 



.Zip. 



.Phone ( ) 
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Part B: Courseware Evaluation Criteria 



Indicate the applicability of each sectio,. to the courseware being evaluated by cl ^ckmg either 

" — A" (applicable) or " N/A" (not applicable) If a section is not applicable, proceed to the next 

section If a section is applicable, check the column that indicates how well the courseware meets 
each criterion. Include any comments. 





YES 


SOME- 
WHAT 


NO 


N/A 


COMMENTS 


1. SUBJECT MASTER* A N/A 




1 Subject matter has educational value. 












2. Student objectives are stated. 












3. Subject matter is accurate. 












4. Subject matter is logically presented. 












5. Subject matter is free of race, ethnic, sex, 
and other stereotypes. 












6 Subject matter is on the level of the 
students. 












7 Information and skills presented are com- 
parable to those used in the home, busi- 
ness, or industry. 












8. Subject matter motivates students to learn 












9 Subject matter is reviewed and 
summarized 












10. Program utilizes the unique capabilities of 
the microcomputer to present the subject 
matter. 












i: TCPUMlPAl DDCCCMTATirtKI A M/A 

n 1 LOnrMIOAL rh CDCiM 1 A 1 IUFM A */A 




1. Program is free of technical problems. 












2. Presentation rate is adequate to maintain 
interest. 












3. Information on the screen is easy to read. 












' 4. Program is free of spelling and grammati- 
cal errors. 












5. Program instructions are easy to follow. 












6. Color increases the instructional value of 
the program. 












7. Audio increases the instructional value of 
the program. 












8. Graphics increase the instructional value 
of the program. 













'Identify the specific basic skills area or vocational program. 
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YES 


SOME- 
WHAT 


NO 


N/A 


COMMENTS 


Ill STUDENT INTERACTION A N/A 




1. Students can use the program with min- 
imal assistance 












2 Students are actively involved in the 
program 












3 Students control the pace of the program 












4 Students can access the program 
"menu(s)" *o change activities 












5 Students are permitted to change answers 












6. Methods of responding correspond to the 
level of the program. 












7 Students' errors of entry are processed so 
that the program continues to run. 












8 Students can access available "help" and 
"hint" options at any time 












9. Students can enter or exit the pro.- ^ as 
desired. 












10 Students control the sequence of the 
program. 












IV. PROGRAM INTERACTION A N/A 




1. Feedback is i' ledidte 












2. Cues and r are provided to assist 
students in mg correctly 












3. Feedback reir . ^es the correct 
responses 












4 Feedback is nonthreatening. 












5. Program helps students understand wrong 
answers. 












6. Prooram oives the cnrrprt anQwpr after » 
reasonable number of tries. 












7. Positive reinforcement is varied. 












8. Program has the ability to branch/loop 
depending upon students' performance. 












9. Feedback is on the level of the student. 












V. STUDENT EVALUATION A N/A 




1 . Evaluation provides a means for measur- 
ing attainment of objectives 












2. Program reports which items were missed 
and which were correct. 
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YES 


SOME- 
WHAT 


NO 


N/A 


COMMENTS 


V STUDENT EVALUATION— Continued 




3 Individual student performance results are 
available to the teacher. 












4. Class performance results are available to 
the teacher. 












5. Program provides for printed copies of 
evaluations. 












6 Test item formats are suited to the material 
being tested. 












7. Test items are clearly stated 












8. Test item bank is provided. 












U DOCUMENTATION A N/A 




1 Documentation is easy to understand. 












2 Documentation is accurate. 












3. Student objectives are stated. 












4 Underlying concepts are outlined. 












5 Skills to be developed are specified 












6. Procedures for integrating the program 
into the curriculum are provided. 












7. Follow-up activities are suggested. 












8 Documentation explains the intended use 
of support materials. 












9 Sufficient information is provided to oper- 
ate the program. 












VII. WORK BEHAVIORS A N/A 




1. Program helps students identify their 
vocational skills. 












2. Program promotes pride *n work. 












3 PrOf 'am nrnmntPQ nroHn^tiuitv/ 
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4. Program encourages good work habits. 












5 Problem solving is encouraged. 












6. Program promotes good human relations 
skills. 












7. Program provides an opportunity for work 
satisfaction and self-fulfillment. 












8. Program encourages creativity. 
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YES 


SOME- 
WHAT 


NO 


N/A 


COMMENTS 


VIII APPLICATION PROGRAMS A N/A 

(to be completed for application programs o;ily) 




1 Program is adaptable to the neecs of the 
student. 












2 Commands are easily remembered 












3 Information is easily manipulated 












4. Corrections are easy to make 










. 


5 Program includes ail necessary variables 












6. Program performs reliably 












7. Program efficiently achieves its intended 
purpose. 












8 Trial data are supplied for learning to run 
the program. 












9 Program provides for use of printer when 
hard copy of information is advantageous 












10. Program moves from operation to opera- 
tion efficiently. 












11. Program is compatible with other applica- 
tion programs. 












12 Program has a supplementary tutorial pro- 
gram available 
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Part C: Courseware Evaluation Summary 



1. SUMMARY COMMENTS 

Identify strengths of the courseware 



Identify weaknesses of the courseware: 



Describe uses of the courseware in an instructionrl setting- 



2. SUMMARY OF SECTION 



Rate the quality of the courseware for each applicable section of this form by checking the 
appropriate column; if not applicable, check N/A. 





YES 


SOME- 
WHAT 


NO 


N/A 


I. SUBJECT MATTER- Content has educational value. 










II. TECHNICAL PRESENTATION: Program is free of maltunc- 
tions. 










III. STUDENT INTERACTION: Students are actively involved with 
the program. 










IV. PROGRAM INTERACTION: Feedback is effectively employed. 










V STUDENT EVALUATION: Evaluation adequately measures 
student progress. 










VI. DOCUMENTATION' Documentation is sufficient to run the 
program. 










VII. WORK BEHAVIORS; Program assists students in developing 
positive work attitudes Lid skills. 










VIII. APPLICATION PROGRAMS- Program performs the task for 
which it is intended. 











3. FINAL RECOMMENDATION 



Check your recommendation for the courseware and explain your reasons below. 

— Highly recommend Recommend with reservations 

— Recommend Do not recommend 



ERIC 
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BASICS ORDER FORM 

BILL AS LISTED BELOW 

a Bill Me 

□ Bill My Agency/Organization on 
Purchase Order No 

□ Purchasing Order Enclosed 

□ Confirming P.O. to Follow 

REMITTANCE 

0 $ U.S. enclosed CK No 

(payable to the National Center for Research in 
Vocational Education) 

□ Payable on receipt of invoice 

BILL TO: 



CHARGE* TO MY CREDIT CARD □ 


□ 




Expiration Date 


Credit Card Number 


mo. yr. 


- $ 


U.S. 


Name on Card (Print or Type) 


Amount 


Authorized Signature 


Date 



Telephone Number 
* Agreeing to pay the sum, set forth to the bank which issued the 
card in accordance with the terms of the credit card. 



SKIP TO: 



Agency 



Name/Title 



Agency 



Name/Title 



Street Address 



Street Address 



City 

Order 
Authorized by 



State 



Zip 



City 



Signature 



State 

OFFICE 

USE ONLY Date 



Zip 



Date 



Authorization 



Order No. 


Title 


Unit 
Price 


Quantity 
Ordered 


Extended 
Price 


SP300A 


The Bridgets Guide (includes) 


$ 75.00 






SF300AA 


Implementation Guide 


10.95 






SP300AB 


Primer of Exemplary Strategies 


11 95 






• 


Improving the Basic Skills of Vocational-Technics! C!u^°nts: 
An Administrator's Guide 


12.00 


H9BB 


^^^^^ 




Integration of Academic and Vocational-Technical Education: 
An Administrator's Guide 


1400 






SP300AC 


Provide for Basic Skills 


7.95 




■ mm 


SP300AD 


Roadsigns from Research (black-line masters) 


14.95 






SP3008 


Introduction to Basics (videocassette— VHS) 


25.00 






SP300C 


Roadsigns from Research (set of 4 posters) 


20.00 






SP300D 


Instructional Program Development (includes) 


50.00 






SP300DA 


Instructional Materials Development 


13.95 






SP300DB 


Supplemental Instructional Resources 


7 95 








Assist Students in Achieving Basic Reading Skills 


5.00 




mwmm 




Assist Students in Developing Technical Reading Skills 


7 50 








Assist Students tn Improving Their Writing Skills 


4.00 








Assist Students in Improving Their Oral Communication Skills 


5.50 








Assist Students, in Improving Their Math Skills 


6,50 






SP300E 


Targeted Teaching Techniques (includes) 


50.00 






5P300EA 


Techniques for Joint Effort: The Vocational-Academic 
Approach (with audiocassette) 


13.95 






SP300E8 


Technique for Management: Time for Learning 


7 50 






SP300EC 


Technique for Remediation: Peer Tutoring (with audiocassette) 


13.95 






SP300ED 


Technique for Computer Use: Software Evaluation 


7 50 






SP300EE 


Technique for Individualization: 
The Academic Development Plan 


9.95 






SP300 

1 


BASICS: Bridging Vocational and Academic Skills (complete set 
at 10% discount) 


198.00 







I 
I 

I 'Obtain additional copies by contacting American Association for Vocational 

Instructional Materials (AAVIM). 120 Driftmier Engineering Center. Athens. 

er|c ja 30602 

I^.-i-i.tt:— n . x 



Sub Total 
(less 



$. 
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- % discount, 
as applicable) Minus 

Total $ 



The Ohio State University 
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